Extracting Moving Objects from a Moving Camera Video Sequence

Extracting moving objects from a video sequence is the first step of many video processing applications, including traffic monitoring and security surveillance. If the camera is fixed in the scene, we can detect moving objects by background subtraction, and many methods have been proposed for generating background images in varying illumination conditions[4, 13]. For images taken by a pan-tilt camera moving around a fixed projection center, we can reduce the problem to the stationary camera case by rectifying the image using the camera control signal [6]. Otherwise, frame-by-frame image mapping is necessary for canceling the background motion caused by the camera motion. Such frame-wise image mapping is based on intensity-based optical flow or feature point matching, using robust estimation techniques, such as LMedS and RANSAC, for avoiding moving object regions [1, 5, 16]. These techniques were originally proposed for the purpose of video data compression [11]. On the other hand, many off-line processing techniques have also been proposed for applications in which time delay is allowed [3, 7, 12], e.g., surveillance systems that first store images which are analyzed later. In this paper, we present an alternative method for such applications. As in existing studies, we assume that the scene is sufficiently far away, the camera motion is small compared with the depth of the scene, and the camera orientation change is also small. Under these assumptions, the image frames are related by homographies. Hence, we can generate a panoramic image of